-
Notifications
You must be signed in to change notification settings - Fork 35.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cli: Detect port errors in rpcconnect and rpcport #29521
cli: Detect port errors in rpcconnect and rpcport #29521
Conversation
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. Code CoverageFor detailed information about the code coverage, see the test coverage report. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please react with 👎 to this comment and the bot will ignore it on the next update. |
6052f49
to
46c91a2
Compare
crACK 46c91a2 |
46c91a2
to
6e779d8
Compare
Thanks for the review @davidgumberg! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK 6e779d8
Testing
Ran ./test/functional/test_runner.py --extended
and got two failures feature_index_prune.py
and wallet_importdescriptors.py --descriptors
. Reran on the base of the PR, 1105aa4 and got both failures, so shouldn't be related. Will investigate further.
Code
I like how you limited the scope of variables introduced in this PR, I wish fewer lines were needed for the added error detection, but I didn't figure out a way to shrink it either.
Some would suggest breaking out to a function, but since it's only used here I agree it's better to inline it.
src/bitcoin-cli.cpp
Outdated
std::optional<std::string> rpcport_arg = gArgs.GetArg("-rpcport"); | ||
if (rpcport_arg.has_value()) { | ||
std::optional<uint16_t> rpcport_port{ToIntegral<uint16_t>(rpcport_arg.value())}; | ||
if (!rpcport_port.has_value() || rpcport_port.value() == 0) { | ||
throw std::runtime_error(strprintf("Invalid port provided in -rpcport: %s", rpcport_arg.value())); | ||
} | ||
|
||
// Use the valid port provided | ||
port = rpcport_port.value(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style: narrow scope of rpcport_arg
since it is only used inside the if-block. (In this case the outer scope ends it anyway, this is just a general rule of thumb).
Naming: rpcport_port
-> rpcport_int
since the former is a bit repetitive and the latter at least describes its purpose somewhat, despite being a bit Hungarian.
std::optional<std::string> rpcport_arg = gArgs.GetArg("-rpcport"); | |
if (rpcport_arg.has_value()) { | |
std::optional<uint16_t> rpcport_port{ToIntegral<uint16_t>(rpcport_arg.value())}; | |
if (!rpcport_port.has_value() || rpcport_port.value() == 0) { | |
throw std::runtime_error(strprintf("Invalid port provided in -rpcport: %s", rpcport_arg.value())); | |
} | |
// Use the valid port provided | |
port = rpcport_port.value(); | |
if (std::optional<std::string> rpcport_arg = gArgs.GetArg("-rpcport")) { | |
std::optional<uint16_t> rpcport_int{ToIntegral<uint16_t>(rpcport_arg.value())}; | |
if (!rpcport_int.has_value() || rpcport_int.value() == 0) { | |
throw std::runtime_error(strprintf("Invalid port provided in -rpcport: %s", rpcport_arg.value())); | |
} | |
// Use the valid port provided | |
port = rpcport_int.value(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good suggestions. I've implemented these changes and added a small comment to increase readability (since operator bool of std::optional is being used).
6e779d8
to
cb4f9fc
Compare
Thank you for the review @cbergqvist! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK cb4f9fc
Testing Methodolgy:
- Ran make, which was successful.
- Ran functional tests, all passed.
- Tried the following
cli
commands in terminal while the daemon was running:
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcconnect=127.0.0.1:notaport help
error: Invalid port provided in -rpcconnect: 127.0.0.1:notaport
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcconnect=127.0.0.1:-1 help
error: Invalid port provided in -rpcconnect: 127.0.0.1:-1
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcconnect=127.0.0.1:65536 help
error: Invalid port provided in -rpcconnect: 127.0.0.1:65536
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcport=notaport help
error: Invalid port provided in -rpcport: notaport
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcport=-1 help
error: Invalid port provided in -rpcport: -1
➜ bitcoin git:(20240229_rpcconnectinvalidportdetection) ✗ bcli -rpcconnect=127.0.0.1:9000 -rpcport=8332 getblockcount
Warning: Port specified in both -rpcconnect and -rpcport. Using -rpcport 8332
448483
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK cb4f9fc
Verified coverage by changing the new C++ code-block to always throw and ran test/functional/interface_bitcoin_cli.py
, confirming the exception message, reset to only PR changes and re-ran test/functional/interface_bitcoin_cli.py
successfully.
Ran ./test/functional/test_runner.py --extended --jobs=16 --timeout-factor=1
and saw the same unrelated tests error out as on the previous version of this PR. Still investigating.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tested ACK. The new error handling code looks good to me upon review. Ran the daemon and gave invalid values for rpcconnect
and rpcport
and the proper error messages came up. The behavior of port 0 being invalid is consistent with src/init.cpp
. Ran interface_bitcoin_cli.py
too and there were no issues with the added tests.
Thank you for reviewing, @marcofleon! |
ACK cb4f9fc The changes in Tested with I also verified that passing bad ports to |
# prefer rpcport over default | ||
assert_equal(BLOCKS, self.nodes[0].cli(f'-rpcport={node_rpc_port}').getblockcount()) | ||
# use default if nothing else available | ||
assert_raises_process_error(1, "Could not connect to the server 127.0.0.1:18443", self.nodes[0].cli("-rpcconnect=127.0.0.1").echo) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In caa5242 "cli: Sanitize ports in rpcconnect and rpcport"
This will fail if you just have a regtest node running independent of the test:
2024-04-22T22:30:22.751000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
File "/home/ava/bitcoin/bitcoin/29521/test/functional/test_framework/util.py", line 122, in assert_raises_process_error
fun(*args, **kwds)
File "/home/ava/bitcoin/bitcoin/29521/test/functional/test_framework/test_node.py", line 821, in __call__
return self.cli.send_cli(self.command, *args, **kwargs)
File "/home/ava/bitcoin/bitcoin/29521/test/functional/test_framework/test_node.py", line 886, in send_cli
raise subprocess.CalledProcessError(returncode, self.binary, output=cli_stderr)
subprocess.CalledProcessError: Command '/home/ava/bitcoin/bitcoin/29521/src/bitcoin-cli' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/ava/bitcoin/bitcoin/29521/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/home/ava/bitcoin/bitcoin/29521/test/functional/interface_bitcoin_cli.py", line 136, in run_test
assert_raises_process_error(1, "Could not connect to the server 127.0.0.1:18443", self.nodes[0].cli("-rpcconnect=127.0.0.1").echo)
File "/home/ava/bitcoin/bitcoin/29521/test/functional/test_framework/util.py", line 127, in assert_raises_process_error
raise AssertionError("Expected substring not found:" + e.output)
AssertionError: Expected substring not found:error: Authorization failed: Incorrect rpcuser or rpcpassword
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for taking a look and trying it out @achow101. Much appreciated.
Good point. Yes, the test would fail if an independent node is running on 18443.
Although not ideal, do you think this would be reasonable since a warning is provided when bitcoind is running independently? (i.e. WARNING! There is already a bitcoind process running on this system. Tests may fail unexpectedly due to resource contention!
)
Alternatively, the assert statement could be removed, but that seems a bit lacking, since it seems good to check for default port usage (even if the current check is limited to the default regtest port).
Could also look into refactoring a bit, but that might lead to touching a bit more code than desirable.
Thoughts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd say it should be avoided to use test outside of the allowed range assigned to this test case. Each test is assigned 12, or so, p2p and rpc ports. I think this test can be dropped, because if the default behavior changed, pretty much every developer would notice quickly anyway?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, that sounds reasonable. Overall, port error checking and associated functional test cases introduce an improvement. Removed the "18443" test and pushed.
cb4f9fc
to
6c17bde
Compare
assert_raises_process_error(1, "Invalid port provided in -rpcconnect: 127.0.0.1:notaport", self.nodes[0].cli("-rpcconnect=127.0.0.1:notaport").echo) | ||
assert_raises_process_error(1, "Invalid port provided in -rpcconnect: 127.0.0.1:-1", self.nodes[0].cli("-rpcconnect=127.0.0.1:-1").echo) | ||
assert_raises_process_error(1, "Invalid port provided in -rpcconnect: 127.0.0.1:0", self.nodes[0].cli("-rpcconnect=127.0.0.1:0").echo) | ||
assert_raises_process_error(1, "Invalid port provided in -rpcconnect: 127.0.0.1:65536", self.nodes[0].cli("-rpcconnect=127.0.0.1:65536").echo) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added this line and it seemed to pass
assert_raises_process_error(1, 'timeout on transient error: Could not connect to the server 127.0.0.1:::', self.nodes[0].cli("-rpcconnect=127.0.0.1::").echo)
looks like SplitHostPort
does not throw an exception if there are two colons and no port provided
I am unsure if that is needed logic in SplitHostPort
I'm seeing the same error when I do
self.nodes[0].cli("-rpcconnect=127.0.0.256")
probably because 255 is the max value
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @kevkevinpal. Good example of why review is valued. Your findings nudged me to add some assert test statements (which otherwise wouldn't be present) to help cover cases involving IPv6 hosts.
We're experiencing the result of the scope of SplitHostPort()
. SplitHostPort()
extracts the host and port. Although the port is validated, host is not (and the function declaration comment header mentions validating the port, but doesn't mention validating the host).
bitcoin/src/util/strencodings.h
Lines 97 to 106 in 7fee0ca
/** | |
* Splits socket address string into host string and port value. | |
* Validates port value. | |
* | |
* @param[in] in The socket address string to split. | |
* @param[out] portOut Port-portion of the input, if found and parsable. | |
* @param[out] hostOut Host-portion of the input, if found. | |
* @return true if port-portion is absent or within its allowed range, otherwise false | |
*/ | |
bool SplitHostPort(std::string_view in, uint16_t& portOut, std::string& hostOut); |
SplitHostPort()
is treating "127.0.0.1::" as an IPv6 host since "::" is used for IPv6 address shortening. SplitHostPort()
detects no port as part of the input string, which would be why Could not connect to the server 127.0.0.1:::18443
is the error provided for src/bitcoin-cli -rpcconnect=127.0.0.1:: getblockcount
(the ":18443" is tacked on the end because it's the default port for regtest).
An enhancement would be to perform a validation of the host received from SplitHostPort()
before bitcoin-cli attempts to contact the host. I'm thinking we would need to first define what is "valid" and what is "invalid" for use with bitcoin-cli. The gist of this kind of checking seems to be present in CNetAddr
. In general, I would think bitcoin-cli shouldn't be overly restrictive in what it lets the user attempt to use as a host.
Lines 157 to 181 in 7fee0ca
[[nodiscard]] bool IsIPv4() const { return m_net == NET_IPV4; } // IPv4 mapped address (::FFFF:0:0/96, 0.0.0.0/0) | |
[[nodiscard]] bool IsIPv6() const { return m_net == NET_IPV6; } // IPv6 address (not mapped IPv4, not Tor) | |
bool IsRFC1918() const; // IPv4 private networks (10.0.0.0/8, 192.168.0.0/16, 172.16.0.0/12) | |
bool IsRFC2544() const; // IPv4 inter-network communications (198.18.0.0/15) | |
bool IsRFC6598() const; // IPv4 ISP-level NAT (100.64.0.0/10) | |
bool IsRFC5737() const; // IPv4 documentation addresses (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24) | |
bool IsRFC3849() const; // IPv6 documentation address (2001:0DB8::/32) | |
bool IsRFC3927() const; // IPv4 autoconfig (169.254.0.0/16) | |
bool IsRFC3964() const; // IPv6 6to4 tunnelling (2002::/16) | |
bool IsRFC4193() const; // IPv6 unique local (FC00::/7) | |
bool IsRFC4380() const; // IPv6 Teredo tunnelling (2001::/32) | |
bool IsRFC4843() const; // IPv6 ORCHID (deprecated) (2001:10::/28) | |
bool IsRFC7343() const; // IPv6 ORCHIDv2 (2001:20::/28) | |
bool IsRFC4862() const; // IPv6 autoconfig (FE80::/64) | |
bool IsRFC6052() const; // IPv6 well-known prefix for IPv4-embedded address (64:FF9B::/96) | |
bool IsRFC6145() const; // IPv6 IPv4-translated address (::FFFF:0:0:0/96) (actually defined in RFC2765) | |
bool IsHeNet() const; // IPv6 Hurricane Electric - https://he.net (2001:0470::/36) | |
[[nodiscard]] bool IsTor() const { return m_net == NET_ONION; } | |
[[nodiscard]] bool IsI2P() const { return m_net == NET_I2P; } | |
[[nodiscard]] bool IsCJDNS() const { return m_net == NET_CJDNS; } | |
[[nodiscard]] bool HasCJDNSPrefix() const { return m_addr[0] == CJDNS_PREFIX; } | |
bool IsLocal() const; | |
bool IsRoutable() const; | |
bool IsInternal() const; | |
bool IsValid() const; |
For now, it might make the most sense to leave the enhancement of host validation to a new PR since this PR is focused on port error detection.
6c17bde
to
867a673
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
reACK 867a673
Inspected git range-diff cb4f9fc~2..cb4f9fc 867a673~2..867a673
.
Passed --extended
functional tests (-feature_dbcrash
, excluded, some others skipped due to configuration). Passed make check
.
self.log.info("Check for ipv6") | ||
have_ipv6 = test_ipv6_local() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(IMO it's slightly cleaner to split the test into --ipv4
and --ipv6
variants similar to rpc_bind.py
, that way it becomes explicit whether that part of the test coverage is being skipped or not. This being more of a parsing-test, I guess it is not on the critical end of the spectrum. Also it's just a minor part of interface_bitcoin_cli.py
so I completely understand the resistance to breaking it out).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @cbergqvist. Good observation.
Looks like rpc_bind.py uses --ipv4
and --ipv6
arguments to run the test exclusively with ipv4 or ipv6 (not both), which at first glance doesn't seem to be desirable for interface_bitcoin_cli.py (dependence on network type is a somewhat small part of the test). I could see there being value in refactoring interface_bitcoin_cli.py to include wider coverage of bitcoin-cli with ipv4 and ipv6, but I'm thinking this sort of refactor would be a good follow-up PR rather than included in the scope of this PR.
In the interim, I did push a minor update to add a log message stating when ipv6 testing is being skipped (and also opportunistically rebased).
867a673
to
b331a60
Compare
Seeing a CI error for arm-linux-gnueabihf-g++ (on a line unchanged in this latest push). Will look into this.
|
🚧 At least one of the CI tasks failed. Make sure to run all tests locally, according to the Possibly this is due to a silent merge conflict (the changes in this pull request being Leave a comment here, if you need help tracking down a confusing failure. |
src/bitcoin-cli.cpp
Outdated
std::optional<uint16_t> rpcport_int{ToIntegral<uint16_t>(rpcport_arg.value())}; | ||
if (!rpcport_int.has_value() || rpcport_int.value() == 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
std::optional<uint16_t> rpcport_int{ToIntegral<uint16_t>(rpcport_arg.value())}; | |
if (!rpcport_int.has_value() || rpcport_int.value() == 0) { | |
port=ToIntegral<uint16_t>(rpcport_arg).value_or(0); | |
if (port == 0) { |
if you treat 0
as invalid, there is no need to also have nullopt be invalid as well, no?
This should also fix the false-positive compiler warning?
An alternative would be to suppress the false-positive warning in the CI task config.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ToIntegral
itself returns a std::optional
unfortunately so the suggested change won't compile as is Super-weird that this error popped up now.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I placed the )
in the wrong place, I think. However, value_or
exists and should be compile-able. See https://en.cppreference.com/w/cpp/utility/optional/value_or
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, that new version should work. Another possible variant.. but no way to know if the compiler becomes happier:
std::optional<uint16_t> rpcport_int{ToIntegral<uint16_t>(rpcport_arg.value())}; | |
if (!rpcport_int.has_value() || rpcport_int.value() == 0) { | |
const uint16_t rpcport_int{ToIntegral<uint16_t>(rpcport_arg.value()).value_or(0)}; | |
if (rpcport_int == 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no way to know if the compiler becomes happier:
If it compiles locally, the code should be fine. If the CI warning persists, it should be disabled in the CI task config. (you can test by pushing, or running locally)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks all for the suggestions. Pushed an update.
Since locally I have x86_64 (and not arm), I tried configuring/compiling with --enable-werror
added to ./configure
(to minimize excess CI churn). Interestingly enough, this didn't result in the same warning as seen on arm CI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No luck with the arm CI again. Will try playing with the specific stage locally more (https://github.com/bitcoin/bitcoin/tree/master/ci).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is a false positive, so it needs to be disabled, like in the other CI tasks.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in #30144
Adds error handling of invalid ports to rpcconnect and rpcport, with associated functional tests.
Adds a warning when both -rpcconnect and -rpcport define a port to be used.
b331a60
to
24bc46c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re-ACK e208fb5. I successfully compiled and built (make check
passed) on my Arm64 machine but the CI error is related to 32-bit machines if I'm not mistaken so might not be relevant.
Either way, the port error detection looks good to me. Clear comments and more readable since my last review. I also successfully ran the functional test. I modified the test using valid ports and mixed -rpcconnect
and -rpcport
configurations to ensure robustness. The test handled everything correctly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
re ACK 24bc46c
Inspected git range-diff 867a673~2..867a673 24bc46c~2..24bc46c
.
Ran same tests as last time (#29521) with same results.
Tried to repro a boiled down version of the mysterious compiler warning but was unable to reproduce it. Here is what I ended up with: https://godbolt.org/z/fh5KcbbeW
(Not sure about the legalese around uploading Bitcoin Core source to Godbolt but just pasting ToIntegral()
shouldn't be that bad).
…task fa73431 ci: Add mising -Wno-error=maybe-uninitialized to armhf task (MarcoFalke) Pull request description: This happens after bd597c3 in many pull requests as a silent merge conflict. For example: * #29720 (comment) * #29521 (comment) * (Probably many undetected, because the CI task was not yet re-run) * ... ACKs for top commit: fjahr: utACK fa73431 fanquake: ACK fa73431 - many fixed with 13.x Tree-SHA512: 6e6ff8dc6f3c6a2abcd04c4203d3468f6e98c1ad3a4da4ad0037a9ee2cbec6bec079a5f778aba0273e38e173849927abcdfcfba7643d08ed66c1168cb89fab08
light code review ACK 24bc46c |
ACK 24bc46c |
Adds invalid port detection to bitcoin-cli for -rpcconnect and -rpcport.
In addition to detecting malformed/invalid ports (e.g. those outside of the 16-bit port range, not numbers, etc.), bitcoin-cli also now considers usage of port 0 to be invalid. bitcoin-cli previously considered port 0 to be valid and attempted to use it to reach bitcoind.
Functional tests were added for invalid port detection as well as port prioritization.
Additionally, a warning is provided when a port is specified in both -rpcconnect and -rpcport.
This PR is an alternate approach to PR #27820 (e.g. SplitHostPort is unmodified).
Considered an alternative to 127.0.0.1 being specified in functional tests, but at first glance, this might need an update to test_framework/util.py (e.g. rpc_url), which might be left to a future PR.